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Abstract Given the accumulating evidence that performance in medical school and 
beyond is related to personality, it is important for research to consider how personality 
assessment can be included as part of the process of selecting medical students. Interviews 
are one way of measuring personality and this study extends prior research investigating 
whether the multiple mini interview (MMI) is related to the five factor model of person¬ 
ality. In contrast to prior results (Kulasegaram et al. in Adv Health Sci Edu 15:415-423, 
2010), examination of MMI scores for 868 applicants to an Australian medical school over 
3 years showed significant uncorrected correlations every year with extraversion (.19, .19, 
.30) and conscientiousness (.20, .22, .25) and with agreeableness in 2 years (.17, .19). 
Investigation of personality at a facet-level revealed differing relationships with the MMI 
within the five factors of personality. MMI scores were also correlated in 2 years (.17, .22) 
with a situational judgment test of interpersonal understanding (UMAT Section 2) but 
were unrelated to tests of logical reasoning ability (UMAT Section 1), non-verbal rea¬ 
soning (Section 3), or past academic performance (Higher School Certificate results). 

Keywords Multiple mini-interview • Personality testing • Five factor model • Medical 
school admissions 


Introduction 

Subsequent to an influential meta-analysis (Barrrick and Mount 1991) identifying signif¬ 
icant relationships between personality and job performance there has been an enormous 
body of research on the impact of personality on work behaviour. Nevertheless, as Musson 
(2009, p. 396) observed, “medical education, in general, seems to lag so far behind other 
fields” in terms of understanding how personality can be used in the selection, training and 
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performance management of medical students and practitioners, and it is only recently that 
personality has become an increasing focus of research in this area. 

The so called “Big Five” model is the most widely accepted taxonomic structure of 
personality variables (Hough and Dilchert 2010) and has been central to much of the 
research linking personality to workplace outcomes. This theory posits that personality is 
structured in a hierarchical fashion with five higher order factors (conscientiousness, 
extraversion, agreeableness, neuroticism and openness to experience), each of which sub¬ 
sumes a number of sub-factors or facets. A review of studies in the medical education context 
conducted between 2000 and 2009 (Doherty and Nugent 2011) concluded that conscien¬ 
tiousness predicts long-term success in medical school (when academic courses include a 
significant clinical component that is likely to require non-cognitive ability for success) and 
together with neuroticism and extraversion, conscientiousness also predicts vulnerability to 
stress in students and interns. These results replicate those found in non-medical settings 
(Barrick et al. 2001) and suggest that, as with many other important roles, the selection of 
medical students would benefit from incorporating personality assessment into the process. 

Personality is a non cognitive construct that is typically measured via self-report 
questionnaire but has also been assessed by other methods such as assessment centers, 
situational judgment tests, and interviews (Hough and Dilchert 2010). Although it appears 
that there is little current use of personality questionnaires for medical student selection, 
the use of interviews is widespread (Albanese et al. 2003), with the recent innovation of the 
multiple mini-interview (MMI) showing significant improvement in the reliability and 
validity of the interviewing process (Reiter et al. 2007). However, while MMIs are pre¬ 
sumed to measure non cognitive constructs, the exact nature of those constructs is the 
subject of ongoing debate. Whether or not they are assessing or in any way related to 
personality was the focus of a recent study by Kulasegaram et al. (2010). The authors 
reasoned that if MMIs were tapping personality, then questionnaire measures of person¬ 
ality could be utilized as a screening test for the MMI. 

There are a number of explanations for the potential link between MMIs and person¬ 
ality. On one hand, stable non cognitive personality traits are likely to influence the 
development of the non cognitive skills and abilities that are often assessed at interview 
(Motowidlo et al. 1997). For example, because extra version reflects sociability, talka¬ 
tiveness and warmth, high scorers are more likely than low scorers to develop skills in 
communication and interpersonal relating, which are typical of the abilities that MMI 
developers attempt to measure (Reiter et al. 2007). Alternatively, MMIs might be like other 
types of selection interview that directly measure aspects of personality. A recent study 
(Huffcutt et al. 2001) categorized 338 interview questions from 47 actual selection 
interviews for non-medical jobs and found that 35% measured personality. Given that 
some MMI stations aim to assess behaviors that evidence whether or not an applicant is 
caring, empathic and collaborative and that such qualities describe the big five factor of 
agreeableness it would not be surprising to find that such stations related to a measure of 
agreeableness. Finally, the possession of certain personality factors could actually affect 
interview performance. For example, conscientious individuals are likely to spend more 
time in preparation for interviews and communicate a sense of self-efficacy and self- 
discipline. In contrast, those high in neuroticism are characterized by anxiety, nervousness 
and self-consciousness, qualities that are likely to impede performance in a competitive 
interview situation. 

However, despite these conceptually intuitive relationships Kulasegaram et al. (2010) 
failed to find any significant correlations between MMI scores and the five factors of 
personality as measured by the short version of the NEO test (McCrae and Costa 1994). 
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This test consists of 60 items (12 for each of the five factors) requiring participants to self- 
report the extent the statements describe their personality. There are three possible reasons 
for why the Kulasegaram et al. (2010) study did not show an association between the MMI 
and personality (other than the fact that there may actually be no relationship), which the 
current research aims to address. 

First, a relatively low sample size (n = 152) drawn from 1 year’s MMI attendees may 
have led to insufficient power. In contrast, our study compares data from three consecutive 
years, with over 250 participants from each year (n = 868 in total). Second, non significant 
results may have been due to factors that act to reduce the reliability of the MMI. As 
reported elsewhere (Griffin and Wilson 2010), we sought to maximize reliability by refining 
the MMI process based on robust findings from the field of organizational psychology (e.g. 
Campion et al. 1997) that indicate interview reliability can be improved by the use of 
anchored rating scales, questions that assess past behaviour, and extensive interviewer 
training. Finally, and more importantly, significant results may have been found if per¬ 
sonality had been examined at a facet-level as well as at the broad five factor level. Although 
the five factors have been instrumental in organizing a model of personality and guiding 
research, more recent work has highlighted the need to investigate the facets of the five 
factors (Hough and Dilchert 2010). Costa and McCrae (1995), early proponents of the five 
factor model, argue that it is not unusual to find variation on facet scores, even for those with 
the same factor scores, as it merely reflects individual differences on related but distinct 
traits. Facet-level research is important because of the evidence showing that the facets 
within a single factor can be differentially related to workplace outcomes despite the fact 
that they relate strongly to each other. Variation of relationships at the facet level can mask 
or reduce the strength of relationships at a factor level. For example, achievement and 
dependability, both facets of conscientiousness, relate differently to job performance 
(Hough 1992). Likewise, half the facets of openness to experience have been found to 
correlate positively to job performance while the other half relate negatively (Griffin and 
Hesketh 2004). In the latter case, this resulted in a near zero correlation between the overall 
factor of openness and job performance. Therefore, in the current study we assess how MMI 
scores relate to both the five factors and their associated facets. 

Because it remains unclear as to the exact nature of the constructs being assessed during 
an MMI, an association found between personality factors and MMI scores would begin to 
define a theory relating personality and non-cognitive ability (Kulasegaram et al. 2010). As 
personality is more strongly related to performance of non-cognitive behaviours (Lievens 
et al. 2009), associations between the MMI and personality would offer an explanation for 
the predictive validity of the MMI found for outcomes that are predominantly non-cog¬ 
nitive in nature, such as clerkship ratings, clinical skills performance, and patient encounter 
ratings (Eva et al. 2009; Reiter et al. 2007). 

Further, if it were found that MMIs can be used to assess personality, development of 
future station content might focus on assessing specific facets of personality. For example, 
the Lievens et al. (2009) study identified the self-discipline facet of conscientiousness as 
particularly important for high academic performance in 6th year medical students so 
scenarios and exercises might be developed that tap this construct. 

Research questions 

1. Which of the big five personality factors are related to the MMI as designed in an 
Australian medical school? 

2. Is there variation in associations with an MMI at a facet-level of the big five factors? 
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Method 

Sample and procedures 

Applicants for entry into an Australian school of medicine were chosen for the MMI 
component of the selection process using a cut-off University Admission Index (UAI; a 
percentile score based on high school academic performance and therefore a similar 
construct to GPA) and ranking on the basis of scores achieved in the Undergraduate 
Medical and Health Science Test (UMAT; a cognitive ability test used by medical schools 
across Australia and New Zealand). The UMAT consists of three sections, each measuring 
different aspects of cognitive ability: Logical Reasoning (Part 1), Understanding People 
(Part 2) and Non-verbal Reasoning Section (Part 3). The psychometric properties of the test 
are reported each year by the test developer, ACER, with some support for its predictive 
validity (Mercer and Puddey 2011). 

After attending the MMI, applicants were invited to complete a research survey, which 
included a measure of personality. In 2006 364 applicants undertook the MMI and 257 
provided personality data (71%), in 2007 336 applicants were interviewed and 287 agreed 
to provide personality data (85%), and in 2008 337 applicants were interviewed with 324 
participating in the research (96%). Ethical approval was given by the University’s HREC. 

Measures 

MMI 

The MMI consisted of nine 8-min stations. A different skill or ability was assessed at each 
station (e.g. verbal communication, empathy, and motivation to study medicine). Although 
station content (i.e. questions, scenario script etc.) varied from year to year, stations were 
designed to assess the same constructs each year. Three ratings were given at each station 
using 5-point scales in 2006 and 2007 and 7-point scales in 2008. The latter was introduced 
in light of interviewer feedback requesting a larger scale. The rating scales included 
anchors that described examples of poor, satisfactory and outstanding performance at 
scores of one, three (or 4 for 7-point scale) and five (or 7 for 7-point scale) respectively. 
Scores were averaged across stations to generate the total score and therefore the differ¬ 
ences in scale length will have no effect on correlation strengths. However, in 2008 one 
station consisted of a psychometric test assessing the ability to identify emotion (i.e. not an 
interview) so the score for this station was not included in the total score for that year. 
Assessment of the reliability of the MMI is a complex issue if the nested nature of the data 
is taken into consideration. However, an indication of the reliability of the total MMI score 
used in this research can be gained from an examination of the coefficient alpha of the 
station scores. Alphas for all 27 scores (three scores for each of nine stations in 2006 and 
2007 but for eight stations in 2008) were .90, .89, and .84 in 2006, 2007 and 2008 
respectively. 

Personality 

A 120-item scale from the International Personality Item Pool (IPIP; Goldberg 1999) was 
used to measure conscientiousness, agreeableness, extraversion, neuroticism and openness 
to experience. The IPIP is a broad-bandwidth, public domain personality inventory that 
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offers a number of scales, including the “NEO PI-R Domains” scale developed to reflect 
the NEO test. These items, which provide factors with good reliability and construct 
validity (Goldberg 1999), have been widely used to measure the big five factors (e.g. 
Griffin and Hesketh 2004; O’Connor et al. 2009) and therefore provide a valid comparison 
to the Kulasegaram et al. (2010) study. The coefficient alphas (a measure of internal 
consistency, with values over .70 considered acceptable levels of reliability (Murphy and 
Davidshofer 1994)) for each of the factors in 2006, 2007 and 2008 were: neuroti- 
cism = .86, .86, and .87; extraversion = .83, .84, and .86; openness = .75, .81, and .78; 
agreeableness = .81, .87 and .87; and conscientiousness = .88, .89, and .91. Furthermore, 
the five factor measures are composed of six facet variables that are each assessed by four 
items (see Table 2 for facet labels). 

Analysis 

In order to compare results with Kulasegaram et al. (2010), the MMI scores were corre¬ 
lated with the personality variables, the UAI (GPA equivalent) and the UMAT (MCAT 
equivalent) using the same p value of .002, which the authors had derived from a Bon- 
ferroni correction. Samples from the 3 years were examined separately as a form of cross- 
validation. In addition, we conducted a multiple regression analysis to investigate the 
influence of the factor-level personality variables on MMI scores, controlling for the 
cognitive ability variables (UAI and UMAT). 


Results 

Table 1 contains the means and standard deviations for the study variables, and Table 2 
reports correlations between the five personality factors, UAI and UMAT scores. 

Relationship of past academic performance and cognitive ability with MMI scores 

The MMI did not relate to measures of past academic performance, logical reasoning 
ability (UMAT Part 1), or non-verbal reasoning (UMAT Part 3) in terms of zero order 


Table 1 Means and standard deviations 



2006 


2007 


2008 


Mean 

SD 

Mean 

SD 

Mean 

SD 

MMI 

3.66 

.50 

3.53 

.52 

4.86 

.70 

UAI 

97.34 

2.99 

97.69 

2.73 

97.80 

2.84 

UMAT part 1 

60.75 

8.00 

62.87 

7.65 

60.23 

6.46 

UMAT part 2 

58.08 

7.19 

57.95 

7.37 

60.33 

6.90 

UMAT part 3 

65.64 

9.27 

62.42 

7.44 

66.62 

10.30 

Neuroticism 

2.33 

.49 

2.27 

.47 

2.33 

.52 

Extraversion 

3.62 

.43 

3.65 

.42 

3.43 

.38 

Openness 

3.57 

.41 

3.55 

.42 

3.55 

.41 

Agreeableness 

4.12 

.37 

4.09 

.43 

4.11 

.43 

Conscientiousness 

3.86 

.47 

3.81 

.48 

3.76 

.49 
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Openness -.16* -.13 -.23* .35* .42* .37* 

Agreeableness -.33* -.43* -.48* .25* .35* .25* .30* .24* .36* 

Conscientiousness —.47* —.53* —.52* .29* .41* .23* .07 .11 .17* .40* .56* .52* 
















Table 3 Correlations between total MMI score and personality across 3 years 
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correlations (see Table 3), but did have a significant relationship with Part 2 of the UMAT, 
which is in the format of a situational judgment test assessing interpersonal understanding. 
Furthermore, when all variables were entered into a regression (Table 4), UAI was sig¬ 
nificant at p < .02 in two of the 3 years as was UMAT Part 2. 

Relationship of personality with MMI scores 

Several significant relationships existed at the factor-level of personality with the MMI. As 
reported in Table 3, extraversion and conscientiousness were correlated with MMI scores 
in all 3 years, and agreeableness was significantly related in two of the 3 years. Because 
these operational validities underestimate the real relationship of an MMI with personality 
to the extent that our test of personality is an unreliable measure, the significant correla¬ 
tions were corrected, using the coefficient alpha reliability value for each factor of per¬ 
sonality. The results are reported in brackets in Table 3. Furthermore, when controlling for 
past academic performance, cognitive ability and all five personality factors (Table 4), 
extraversion was significantly related to MMI scores in all 3 years and conscientiousness in 
2 years (at p < .05). 

Examination of personality at a facet-level (Table 3) revealed variation within the 
factors. Although neuroticism and five of its facets were unrelated to MMI performance, 
the facet of self-consciousness was significantly negatively correlated with the MMI every 
year, indicating the more self-conscious a person the poorer they performed. Four of the six 
facets of extraversion and conscientiousness were consistently correlated with MMI scores, 
but two facets (excitement seeking and cheerfulness under extraversion, and orderliness 
and dutifulness under conscientiousness) were unrelated. Relationships within the domains 
of agreeableness and openness were more variable, although altruism was significantly 
correlated in 2 years. Altruism was actually a characteristic assessed at one station of the 
MMI. 


Discussion 

Unlike an initial investigation into the relationship between personality and MMI perfor¬ 
mance (Kulasegaram et al. 2010), the current study was able to demonstrate a significant 
relationship between at least two personality factors and the MMI, and also illustrated how 
examination of personality at a facet-level provides a clearer understanding of what stable 
traits influence MMI performance. 

The finding that MMIs tapped the constructs of extraversion and conscientiousness 
indicates that those who perform well on an MMI are likely to enjoy being with people, are 
full of energy, enthusiastic, action-oriented individuals and are likely to strive hard to 
achieve excellence, be reliable and hard-working, and persist even when a task is difficult 
or unpleasant. This is particularly important given the large body of literature indicating 
the effect extraversion and conscientiousness have on significant workplace behaviors and 
outcomes. Extraversion has been demonstrated to relate strongly to leadership and team¬ 
work (Barrick et al. 2001) while conscientiousness predicts job performance, training 
performance, and learning in educational settings (Barrick et al. 2001). More specifically, a 
recent study (Lievens et al. 2009) showed that these two factors are significant predictors of 
successful academic performance during the final years of medical school when clinical 
skills are an important component of success. Although the observed and corrected 
validities were not strong, they are typical of those found in field research, particularly in 
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selection context when subjective ratings of performance are used (Barrrick and Mount 
1991). 

The question remains as to why the MMI was not related to the other three personality 
factors. A possible explanation comes from research in social psychology on the accuracy 
of judgments of other people’s behavior, which has consistently shown that not all per¬ 
sonality traits are equally observable in short social interactions. In particular, strangers 
(for example, MMI interviewers) are most able to make reasonably accurate judgments 
about another person’s extraversion and conscientiousness as opposed to their neuroticism, 
openness or agreeableness (Connolly et al. 2007). 

Furthermore, the pattern of results related to the personality facets were remarkably 
similar to Lievens et al.’s (2009) results, which linked the same facets to performance in 
medical school. These authors showed that within the factor of conscientiousness for 
example, the more proactive traits (self-efficacy, achievement, and self-discipline) had 
stronger correlations with student grades than the inhibitory facets (order, dutifulness, and 
cautiousness). Our study found that the inhibitory facets were less strongly related to the 
MMI, providing some initial evidence that performance in MMIs relate to the same 
qualities that relate to performance in the final years of medical school. MMI developers 
might therefore focus on developing station content that activates behavior representing 
these specific facets. 

The significant findings in the current study could have been due to improved power from 
the higher sample sizes that represented a substantial proportion of the interviewees from 
each of 3 years’ MMI. It is also possible that the raters on these MMIs were more likely to be 
accurate at assessing applicants because we reduced their cognitive load through use of 
anchored rating scales and because the three ratings made at each station were very specific to 
the construct being measured at the station (see Huffcutt and Culbertson 2011). 

This study supports the prior results indicating that MMIs are not a direct measure of 
cognitive ability (Kulasegaram et al. 2010). However, when controlling for personality, those 
with higher UAI scores did perform better than those with lower scores as did those with higher 
“interpersonal understanding” (as measured by the situational judgment tests constituting 
Book 2 of the UMAT exam). The latter result indicates that those with higher interpersonal 
understanding may perform well in contexts such as interviews—drawing on that under¬ 
standing could help them interact successfully with interviewers and grasp what is required of 
good performance in a station. Nevertheless, the mounting evidence that MMIs assess non 
cognitive qualities highlights their unique contribution to medical student selection. Clearly a 
medical practitioner requires both cognitive and non-cognitive skills for good job performance 
and therefore selection processes should include assessments of both domains. 

As with most of the research on MMIs, this study was limited to an MMI at one 
university in one country, potentially affecting it’s applicability to other contexts. Whilst 
the results have begun to address the need to identify the constructs underlying the MMI, it 
is quite possible that MMIs across universities differ in what they are measuring. The rate 
of participation in the study was relatively high (71-96% of all interviewees), but it was 
nevertheless a volunteer Australian sample providing self-reported personality data who 
may differ from their counterparts in other countries. 


Conclusion 

In contrast to Kulasegaram et al. (2010), the results of the current study show that there is 
an association between MMI scores and personality. As a result ongoing work on MMIs 
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might consider the development of station content that specifically activates non-cognitive 
traits that relate to the facets within the domains of conscientiousness and extraversion that 
are considered of importance in medicine. Nevertheless, the strength of the relationship 
suggests that MMIs and personality questionnaires are not interchangeable so we would 
therefore not recommend that personality tests be used as a screening tool for MMIs. It 
remains for future research to ascertain if both contribute independently to the prediction of 
success in medical school and beyond. 
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